Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

No commit reply verify #15040

Merged
merged 2 commits into from
Oct 26, 2024
Merged

No commit reply verify #15040

merged 2 commits into from
Oct 26, 2024
+337 −9

Conversation

areshand
Copy link
Contributor

@areshand areshand commented Oct 21, 2024 •
edited
Loading

Description

This is a basic no commit replay verify with basic task divide and conquer

currently, only 14K TPS. Probably can make it a pipeline or add cache on top of the DbstateView to speed up. But not plan to do it in this PR.

How Has This Been Tested?

manual test with Cli

Key Areas to Review

Type of Change

  • New feature
  • Bug fix
  • Breaking change
  • Performance improvement
  • Refactoring
  • Dependency update
  • Documentation update
  • Tests

Which Components or Systems Does This Change Impact?

  • Validator Node
  • Full Node (API, Indexer, etc.)
  • Move/Aptos Virtual Machine
  • Aptos Framework
  • Aptos CLI/SDK
  • Developer Infrastructure
  • Move Compiler
  • Other (specify)

Checklist

  • I have read and followed the CONTRIBUTING doc
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I identified and added all stakeholders and component owners affected by this change as reviewers
  • I tested both happy and unhappy path of the functionality
  • I have made corresponding changes to the documentation

@trunk-io Trunk.io
Copy link

trunk-io bot commented Oct 21, 2024 •
edited
Loading

⏱️ 3h 12m total CI duration on this PR
Slowest 15 Jobs Cumulative Duration Recent Runs
execution-performance / single-node-performance 49m 🟩 🟩 
rust-smoke-tests 28m 🟩 
rust-cargo-deny 26m 🟩 🟩 🟩 🟩 🟩 (+11 more)
check-dynamic-deps 22m 🟩 🟩 🟩 🟩 🟩 (+12 more)
execution-performance / test-target-determinator 9m 🟩 🟩 
semgrep/ci 7m 🟩 🟩 🟩 🟩 🟩 (+12 more)
general-lints 7m 🟩 🟩 🟩 🟩 🟩 (+11 more)
rust-doc-tests 5m 🟩 
test-target-determinator 5m ⬜ 🟩 
check 4m ⬜ 🟩 
file_change_determinator 3m 🟩 🟩 🟩 ⬜ 🟩 (+12 more)
rust-move-tests 2m 🟩 
rust-move-tests 2m 🟩 
rust-move-tests 2m 🟩 
rust-move-tests 2m 🟩 

🚨 1 job on the last run was significantly faster/slower than expected

Job Duration vs 7d avg Delta
execution-performance / single-node-performance 25m 16m +54%

settings ⋅ feedback ⋅ docs ⋅ learn more about trunk.io

@areshand areshand force-pushed the no_commit_reply_verify branch 7 times, most recently from 90471ff to dd9b08d Compare October 22, 2024 21:19
@areshand areshand marked this pull request as ready for review October 22, 2024 21:24
msmouse
msmouse reviewed Oct 22, 2024
View reviewed changes
storage/db-tool/src/replay_on_archive.rs Outdated Show resolved Hide resolved
storage/db-tool/src/replay_on_archive.rs Outdated Show resolved Hide resolved
@areshand areshand force-pushed the no_commit_reply_verify branch 3 times, most recently from 9afb2a9 to 8480235 Compare October 22, 2024 22:36
@areshand areshand requested a review from msmouse October 22, 2024 22:43
@areshand areshand force-pushed the no_commit_reply_verify branch 3 times, most recently from f02a2cc to 9818533 Compare October 25, 2024 22:09
@areshand areshand force-pushed the no_commit_reply_verify branch from 9818533 to e6c7650 Compare October 25, 2024 22:21
@areshand areshand requested a review from msmouse October 25, 2024 22:23
@areshand areshand enabled auto-merge (rebase) October 25, 2024 23:13
@github-actions GitHub Actions

This comment has been minimized.

Sign in to view
@github-actions GitHub Actions

This comment has been minimized.

Sign in to view
@github-actions GitHub Actions

This comment has been minimized.

Sign in to view
@github-actions GitHub Actions
Copy link
Contributor

✅ Forge suite realistic_env_max_load success on d419f7e70eaf337d1eb9d5054928c5072a14f8ad

two traffics test: inner traffic : committed: 14052.53 txn/s, latency: 2828.72 ms, (p50: 2700 ms, p70: 2700, p90: 3000 ms, p99: 3300 ms), latency samples: 5343080
two traffics test : committed: 100.06 txn/s, latency: 1930.62 ms, (p50: 1800 ms, p70: 2200, p90: 2300 ms, p99: 8900 ms), latency samples: 1760
Latency breakdown for phase 0: ["MempoolToBlockCreation: max: 1.751, avg: 1.238", "ConsensusProposalToOrdered: max: 0.328, avg: 0.299", "ConsensusOrderedToCommit: max: 0.368, avg: 0.358", "ConsensusProposalToCommit: max: 0.667, avg: 0.657"]
Max non-epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 1.15s no progress at version 2695464 (avg 0.21s) [limit 15].
Max epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 8.42s no progress at version 2695462 (avg 8.42s) [limit 15].
Test Ok

@github-actions GitHub Actions
Copy link
Contributor

✅ Forge suite framework_upgrade success on b29f09f57e898d8d211c8bc3e303f6e50bba2266 ==> d419f7e70eaf337d1eb9d5054928c5072a14f8ad

Compatibility test results for b29f09f57e898d8d211c8bc3e303f6e50bba2266 ==> d419f7e70eaf337d1eb9d5054928c5072a14f8ad (PR)
Upgrade the nodes to version: d419f7e70eaf337d1eb9d5054928c5072a14f8ad
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1146.95 txn/s, submitted: 1149.41 txn/s, failed submission: 2.46 txn/s, expired: 2.46 txn/s, latency: 2674.93 ms, (p50: 2400 ms, p70: 2800, p90: 4600 ms, p99: 6300 ms), latency samples: 102560
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1179.10 txn/s, submitted: 1181.39 txn/s, failed submission: 2.29 txn/s, expired: 2.29 txn/s, latency: 2870.68 ms, (p50: 2400 ms, p70: 2800, p90: 5400 ms, p99: 6900 ms), latency samples: 92840
5. check swarm health
Compatibility test for b29f09f57e898d8d211c8bc3e303f6e50bba2266 ==> d419f7e70eaf337d1eb9d5054928c5072a14f8ad passed
Upgrade the remaining nodes to version: d419f7e70eaf337d1eb9d5054928c5072a14f8ad
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1108.08 txn/s, submitted: 1111.18 txn/s, failed submission: 3.10 txn/s, expired: 3.10 txn/s, latency: 2634.21 ms, (p50: 2400 ms, p70: 2700, p90: 4100 ms, p99: 5700 ms), latency samples: 100000
Test Ok

@github-actions GitHub Actions
Copy link
Contributor

✅ Forge suite compat success on b29f09f57e898d8d211c8bc3e303f6e50bba2266 ==> d419f7e70eaf337d1eb9d5054928c5072a14f8ad

Compatibility test results for b29f09f57e898d8d211c8bc3e303f6e50bba2266 ==> d419f7e70eaf337d1eb9d5054928c5072a14f8ad (PR)
1. Check liveness of validators at old version: b29f09f57e898d8d211c8bc3e303f6e50bba2266
compatibility::simple-validator-upgrade::liveness-check : committed: 11579.69 txn/s, latency: 2884.05 ms, (p50: 1700 ms, p70: 1800, p90: 4400 ms, p99: 25400 ms), latency samples: 433460
2. Upgrading first Validator to new version: d419f7e70eaf337d1eb9d5054928c5072a14f8ad
compatibility::simple-validator-upgrade::single-validator-upgrading : committed: 6568.22 txn/s, latency: 4320.56 ms, (p50: 5000 ms, p70: 5200, p90: 5400 ms, p99: 5500 ms), latency samples: 120520
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 6834.83 txn/s, latency: 4735.50 ms, (p50: 5200 ms, p70: 5300, p90: 5400 ms, p99: 5500 ms), latency samples: 229960
3. Upgrading rest of first batch to new version: d419f7e70eaf337d1eb9d5054928c5072a14f8ad
compatibility::simple-validator-upgrade::half-validator-upgrading : committed: 6051.27 txn/s, latency: 4670.56 ms, (p50: 5300 ms, p70: 5500, p90: 5900 ms, p99: 6100 ms), latency samples: 114100
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 6679.59 txn/s, latency: 4812.02 ms, (p50: 5000 ms, p70: 5200, p90: 6700 ms, p99: 6900 ms), latency samples: 224500
4. upgrading second batch to new version: d419f7e70eaf337d1eb9d5054928c5072a14f8ad
compatibility::simple-validator-upgrade::rest-validator-upgrading : committed: 7381.86 txn/s, latency: 3761.13 ms, (p50: 3600 ms, p70: 4400, p90: 5800 ms, p99: 6900 ms), latency samples: 130200
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 7650.08 txn/s, latency: 4110.82 ms, (p50: 4100 ms, p70: 5000, p90: 5700 ms, p99: 6400 ms), latency samples: 259940
5. check swarm health
Compatibility test for b29f09f57e898d8d211c8bc3e303f6e50bba2266 ==> d419f7e70eaf337d1eb9d5054928c5072a14f8ad passed
Test Ok

@areshand areshand merged commit f3466b6 into main Oct 26, 2024
45 of 48 checks passed
@areshand areshand deleted the no_commit_reply_verify branch October 26, 2024 04:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@grao1991 grao1991 grao1991 approved these changes

@msmouse msmouse msmouse approved these changes

@lightmark lightmark Awaiting requested review from lightmark lightmark is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@areshand @msmouse @grao1991